Fusion of Data from Head-Mounted and Fixed Sensors

A methodology is developed to explicitly fuse sensor data from a combination of fixed and head-mounted sensors, in order to improve the registration of objects in an augmented reality system. The methodology was applied to the analysis of an actual experimental augmented reality system, incorporating an optical see-through head-mounted display, a head-mounted CCD camera, and a fixed optical tracking sensor. The purpose of the sensing system was to determine the position and orientation (pose) of a movable object with respect to the head-mounted display. A typical configuration was analyzed and it was shown that the hybrid system produces a pose estimate that is significantly more accurate than that produced by either sensor acting alone. Using only the fixed sensor, the maximum translational error in the location of an object with respect to the head-mounted display in any direction was 27.4 mm (corresponding to a 97% confidence interval). Using only the head-mounted sensor, the maximum translational error in any direction was 20.4 mm. By combining data from the two sensors, the maximum translational error was reduced to 3.3 mm. In order to fuse the pose estimates, the uncertainties are explicitly calculated, in the form of covariance matrices. A capability was also developed to visualize the uncertainties as 3D ellipsoids.